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Abstract —This paper considers optimization of power and 
delay in a time-varying wireless link using rateless codes. The 
link serves a sequence of variable-length packets. Each packet 
is coded and transmitted over multiple slots. Channel conditions 
can change from slot to slot and are unknown to the transmitter. 
The amount of mutual information accumulated on each slot 
depends on the random channel realization and the power used. 
The goal is to minimize average service delay subject to an 
average power constraint. We formulate this problem as a frame- 
based stochastic optimization problem and solve it via an online 
algorithm. We show that the subproblem within each frame is 
a simple integer program which can be effectively solved using 
a dynamic program. The optimality of this online algorithm is 
proved using the frame-based Lyapunov drift analysis. 

I. Introduction 

Consider a slotted time system where a single source 
transmits a sequence of data packets to a receiver over a time 
varying channel. During each time slot, the source transmits 
a portion of the mutual information needed for the receiver 
to decode the current packet. This is done by choosing the 
amount of power on each transmission. The source does not 
know the current channel state when it makes this decision. 
The chosen power and the random channel state determine the 
amount of mutual information that is received. Transmission of 
that packet is finished when the receiver accumulates enough 
mutual information to decode. This technique is usually re¬ 
ferred to as mutual information accumulation (im-ci) and 
can be implemented through rateless coding schemes such as 
Raptor codes and Fountain codes (ll3l-P1l). 

Our goal is to design a transmission algorithm such that the 
time average delay of the sequence of packets is minimized 
and a time average power constraint is satisfied. Specifi¬ 
cally, we consider an online setting where both the channel 
conditions and the packet lengths could vary according to 
some underlying probability distributions. This problem is 
challenging for two reasons: (i) Decoding times can vary 
randomly for each packet, (ii) The power spent during the 
current time slot affects the desired power constraint and also 
influences how much information is left to transmit for that 
packet in future slots, creating complex dependencies over 
different slots. Fortunately, the system contains an underlying 
Markov renewal structure that is exploited in this paper. 

In this paper, we use renewal reward theory (e.g. Q) to¬ 
gether with Lyapunov optimization (e.g. ||6l) to develop an on¬ 
line algorithm for this problem. Previous work Q introduces 
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a general framework for applying Lyapunov optimization to 
renewal systems through the idea of dividing the time line 
into renewal frames and solving a subproblem within each 
renewal frame. Works in ID and 191 analyze the performance 
of frame-based Lyapunov optimization for a file downloading 
problem and a multiclass M/G/1 queue problem, respectively. 
In both papers, the subproblem within each frame is effectively 
solved, which is the key to the success of the whole algorithm. 
In this paper, we show that the subproblem within each frame 
is an integer program which can be effectively solved using a 
dynamic program. 

Prior work on mutual information accumulation in 
GoliniElIIl focus on a network with fixed channel con¬ 
ditions. Works in Co) and ca consider the problem of 
minimum energy accumulative broadcast and show that it 
is NP-complete. Works El and 112 consider the problem 
of optimal cooperative routing in a multi-hop network and 
develop linear program based formulations. Applications to 
time-varying channel conditions are considered in m and 
M- All these previous works only consider the optimal 
transmission of a single packet. Our work differs from these 
works in that we consider a sequential transmission scenario 
with a time average power constraint. 

The contributions of the paper are as follows: First, we 
formulate this power constrained minimum delay problem as 
a frame-based stochastic optimization problem, where each 
frame is the time period for transmitting one single packet. 
Second, we propose a provably optimal frame-based online 
algorithm where an integer program is solved during each 
frame. We show that the integer program can be effectively 
solved through dynamic programming. A nice feature of the 
resulting algorithm is that, while it assumes knowledge of the 
channel probability distribution, it does not require knowledge 
of the file size probability distribution. Further, the power 
constraint is shown to hold even if there is a mismatch between 
the actual channel probability distribution and the assumed 
distribution. 

IT Problem Formulation 

In this section, we present our system model and the 
delay minimization problem. Consider a slotted time single 
hop network where one source is transmitting a sequence 
of packets to a receiver with time varying channel states. 
The transmitter uses a rateless coding scheme where the 
receiver accumulates mutual information about each packet 
until it can successfully decode. The goal is to minimize 
the time average packet delay subject to an average power 
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constraint on the source. For simplicity of analysis, we assume 
that the source’s only significant power expenditure lies in 
transmissions, and coding/decoding yields negligible power 
and time consumption. 

Let (5 > 0 be a fixed amount of mutual information, called 
a data unit. Let L[f] be the length of the /-th packet, which is 
a positive integer number of data units associated with packet 
/. We assume that L[f] is i.i.d. over / G {0,1,2, ••■} and 
takes values in a finite set £. Also, assume that the source 
uses ideal rateless codes for transmission so that the mutual 
information collected by the receiver from different slots for 
the same packet add up with each other. Then, the receiver can 
decode packet / if and only if it accumulates at least L[f] data 
units for that packet. Finally, we assume that the source cannot 
transmit any new packet until the transmission of the previous 
packet is finished. 

The source has different power spectral density (PSD) op¬ 
tions to take, which is in units of joules/slot/Hz. Let P{t) G V 
be the option taken by the source at slot t. We assume 
V = {Pi,P 2 r '' tP\V\} finite. Without loss of generality, 
further assume 0 < Pi < P 2 < < P\v\- The channel 

power gain at time slot t is denoted as a{t). We assume 
a{t) > 0 is i.i.d. over slot and takes values in a finite set 
A = {ai,a 2 ,--- :Ci\A\} with probability distribution 
They satisfy 

1^1 

4>{ai) > 0 , = 1 . 

i=l 


Then, the mutual information transmitted during a single 
time slot t can be expressed as K{a{t), P{t)), which is 
also a positive integer number of data units. We make no 
assumption on the specific form of K{a{t), P{t)) except 
that it must be nondecreasing with respect to both a{t) and 
P{t). For example, if the source uses fountain codes, then by 


log2 (1 


No ) 


Shannon’s formula, K{a{t), P{t)) = 
where Nq/2 denotes the PSD of the white noise process. 

For the ease of algorithm development, we further segment 
the slotted timeline into frames. Define frame / as the time 
period (in units of slots) of transmitting packet /, with starting 
slot denoted as tf and frame length denoted as T[f] = tf+i — 
tf. Frame / ends on the first slot in which the total mutual 
information for that frame exceeds L[f], which is 


E 


K{a{t),P{t))>L[f]. 


We make the following assumption regarding the random 
processes {L[/]}^o {a(f)}^o: 

Assumption 1: The distribution of packet length process 
{P[f]}'fLo unknown to the source but the source can 
observe L[f] at the beginning of each frame. The process 
{a(f)}“g is unobservable, meaning that the source cannot 
see a{t) before transmission at slot t, but the source has the 
knowledge of distribution 4>{ai), cti G A. 

Furthermore, at the end of slot t, the transmitter provides a 
feedback signal so that the source effectively knows a{t) and 
thus how much mutual information has been transmitted. The 


goal is to choose the PSD allocation P{t) at each time slot 
so as to minimize the average delay (in units of slots/packet) 
subject to a time average power constraint: 


min 


s.t. 


lim 

F—^OO 


F 


lim 

F—foo 


Z^/=0 ^t^tf 


p{t) 




<13, 


where /3 is a positive constant. An equivalent formulation 
which does not include a ratio of time averages in the 
constraint is as follows: 


F^ca t ^' 

/=0 

S.t. lim p E E - /3) < 0. (2) 

f=0 t=tj 

For the rest of the paper, we shall consider ([TJ-© instead. We 
also need the assumption that the minimal PSD allocation Pi 
is greater than 0 but small enough. This ensures that the frame 
length T[f] is finite for any policy, and the problem described 
is feasible. 

Assumption 2: The minimal PSD option Pi satisfies 0 < 

Pi <13. 

Furthermore, let 


Tmax — max L, 

L&C 

Kmin = min K{a,Pi). 

Here Lmax stands for maximum packet length and iTmin 
stands for minimum per slot mutual information transmission. 
Notice that since K{a,P) is a positive integer for any a and 
P, we have A"min > 1- 

III. An Online Algorithm 

In this section, we propose an online algorithm which adapts 
to the time varying packet lengths and channel power gains. 


A. Proposed algorithm 

The proposed algorithm operates over frames, and features 
a “virtual queue” technique treating the time average power 
constraint. Define a virtual queue Q[f], which has initial 
condition Q[0] =0 and update equation: 

L+1-1 'I 

Q[f]+ (Ft)-I3), o\. (3) 

t=tf ) 

The proposed algorithm then runs in the following two steps 
with a trade-off parameter L > 0: 

« At the beginning of frame /, observe Q[f], L[f], and 
choose a sequence of PSD allocations so as to solve the 


Q[f -I- 1] = max ■ 
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following optimization problem: 


mm 

p{t)er 


E 

E 

1_ 

_1 


t — tf 

.1-1 

- 


s.t. K{a{t),P(t)) > L[f] 


(4) 


(5) 


where for any allocation P{t) S V, the function 
Rf{P(t)) is defined as Rf{P{t)) = V + Q[f]{P{t) - j3), 
and the expectation is taken with respect to the random 
channel gain a{t). 

• Update Q[f] according to Q at the end of each frame. 
Section |IV] shows the above algorithm structure produces 
a near optimal solution, where proximity to optimality is 
determined by the V parameter. The following section shows 
how to implement each state of the algorithm, that is, how to 
solve the problem (|4]i-(|5]l. 


B. Solve @-(151) via dynamic program under static channel 
gain 

In this section, we show how to solve @-@ during each 
frame under static channel gain, i.e. a{t) = ap, Vf, where ap 
is a positive constant. Then, we show how to generalize the 
algorithm to random channel case in the next section. 

Given Q[f] and L[f], there is no randomness in the formu¬ 
lation @-®. Thus, we can simplify it as follows: 


min 

P{t)GV 


s.t. 


U+i-i 

E 


t—tf 


E Kiao,Pit))>L[f]. 


t — t f 


( 6 ) 

(7) 


Next, we distinguish between the following two cases: 

1) Case 1: Suppose there exists some option Pj^ G V 
such that RfiPjf,) < 0, then, since both Rf{P(t)) and 
K{ao, P{t)) are increasing with P{t), the best decision 
is always choosing the smallest power allocation (i.e. 
P(t) = Pi) till the end of the frame. 

2) Case 2: Suppose Rf{Pj) > 0, VPj G V. Then, we 
can interpret above problem in the following way: For 
each PSD option Pj G V, there is a corresponding 
penalty Rf{Pj) and a weight K{ao,Pj). The goal is 
to minimize the total penalty subject to the constraint 
that the total weight should be at least L[f]. Since there 
are only finite options in P, (@-0 can be written as 
the following integer program problem: 


min 

^3, jelb-J-PI} 


s.t. 


I-PI 

i=i 

I-PI 

^K{ao,Pj)xj > L[f], Xj G N. 
j=i 


Notice that both K{ao,Pj) and L[f] are positive inte¬ 
gers. This is similar to a classical Knapsack problem. 


A standard technique solving this type of problems is 
dynamic program (see ifTsl for related results). Define 
the state variable fe, fc G Z and define m[k], fc G Z as 
the minimum value achieved by solving the following 
problem: 


min 

^ 3 , jelb'-JPI} 


s.t. 


\p\ 

E/ 

i=l 

I-PI 

''^^K{ao,Pj)xj > k, Xj G N. 

i=l 


(8) 

(9) 


Define m[k] = 0, Vfc < 0, then, we progessively 
compute m[k] for each fc G {1, 2, • • • , L[/]} as follows: 


m\k] = min RfiPj) + m\k — Klao.Pj)], 

yk > 0 . 


The proof of optimality is similar to that of knapsack 
problem which can be found, for example, in chapter 6 

of Ida. 


C. Extension to random channel gain 

In this section, we solve @-0 under random channel gains. 

Again, we distinguish between the following two cases: 

1) Suppose there exists some option Pj^ G V such 
that Rf{Pj^) < 0. Then, since both Rf{P{t)) and 
K{a{t), P{t)) are increasing with P{t) for any a{t) G 
A, the best decision is always choosing the smallest 
power allocation (i.e. P{t) = Pi) until the end of the 
frame. 

2) Suppose Rf{Pj) > 0, VPj G V. Then, in a similar 
way as before, we get a randomized integer program, 
which can still be solved by dynamic program. Define 
k, fc G Z as the state variable and define m[k], fc G Z as 
the minimum value achieved by the following problem: 



*/+i-i 


min E 
P{t)eP 

vT[f]+Q[f] E im-p) 

t=tf 

Q[f],m 



(10) 

*/+i-i 


s.t. E 7f(P(f), a(f)) > k 

(11) 


t=tf 

Define m[k] = 0, V/c < 0, then, we progessively 
compute m[k] for each fc G {1, 2, • • • , L[/]} as follows 


1-41 

m\k] = min RfiPi) + > (h(ai)m\k 

1—1 

-K{a„P,)iyk>0, (12) 

The proof that ([12]) indeed leads to optimal policy during 
each frame is similar to that of a standard proof of 
optimality for a dynamic program (see M for more 
details). For completeness, we provide a proof using 
induction in Appendix lAl 
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IV. Performance Analysis 

In this section, we prove that the online algorithm in Section 
IlII-AI satisfies the power constraint and achieves the near 
optimal throughput (with 0{\/V) optimality gap). We first 
show that the virtual queue Q[f] is bounded, which implies 
that the time average of “arrival process” 
than or equal to that of “service process” l3T\f], and thus the 
power constraint is satisfied. Then, we prove that the algorithm 
gives near optimal performance by Lyapunov drift analysis. 


A. Average power constraint via queue bound 

Lemma 1: If there exists a constant C > 0 such that Q[f] < 
C, then, it follows for any F > 0, 


F-ltf+i-l 

/=0 t=t/ 


C 

F’ 


thus. 


F—l tf+i — 1 


limsup p X! X! - /3) < 0- 


/=o t=tf 


Proof: From Q, we know that for each frame /: 

Q[f + i]>Q[f]+ 

t—t f 

Rearranging terms gives: 

tf+i-i 

E iPit)-p)<Q[f+n-Q[f] 

t—t f 

Fix F > 0. Summing over / S {0,1, • • • , F — 1} gives: 

F-l tf+i-1 

E E iPit)-p)<Q[F]-Qm<c, 

/=0 t=tf 

which implies: 

F-ltf+^-l 

jE E 

f=o t=tf 

Taking F ^ oo gives the desired result. ■ 

The next lemma shows that the queue process deduced from 
proposed algorithm is deterministically bounded. 

Lemma 2: If (5[0] = 0, then, the proposed algorithm gives 
for all / e N+, 


Q[f] < max 


^ ^ 1 

-^max 

\P-Pl 

-^min 


■{P\v\-I3), 



where \x~\ denotes the ceiling function rounding up towards 
the smallest integer greater than x. 

Proof: First, consider the case when P\-p\ < /3. From 
(|3ll it is obvious that Q[f] can never increase. Thus, Q[f] = 
0, V/ G N+. 


Next, consider the case when P|-p| > /?. It is obvious that 
the frame length T[f] is upper bounded by 




. We then 


prove the assertion by induction on /. The result trivially holds 
for / = 0. Suppose it holds at / = / for I G N+, so that: 


Q[l]< 


V 


-Pi 




K„ 


{P\v\-P)- 


We are going to prove the same holds for f = I 
are two further cases to be considered: 

1) Q[l] < In this c^se we have by Q: 

Q[l + l] < Q[l]+T[l]{P\-p\-P) 


1. There 


< 


V 


-Pi 




■(PPI-^)- 


/3-Pi ^ - /3-Pi + 




• {P\v\ — P)- In this 


case, we have by first half of induction hypothesis. 


Q[l]Pi + V -PQ[l] < 0. 

By the first case in Section ITlI-CI the source will choose 
power allocation P{t) = Pi, Vf G — 1]. Since 

Pi < /3 by assumption |2] it follows from the queue 
updating rule Q that the queue will increase no more 
at the end of frame 1. By the second half of the induction 
hypothesis. 

This finishes the proof. ■ 

Combining above two lemmas gives the result that the pro¬ 
posed algorithm satisfies the power constraint. Notice that 
above result is a sample path result which only assumes 
ATmin > 0, P — Pi >0 and probability pp) G [0,1]. Thus, the 
algorithm meets the average power constraints even if it uses 
incorrect values for these parameters. 


K„ 


{P\p\ - P)- 


B. Optimization over randomized stationary algorithm 

Suppose the server knows the probability distribution of 
packet length. Then, consider the following class of algorithm: 
At the beginning of each frame /, after observing the packet 
length L[f], the source independently selects a sequence of 
power allocation P*{t),t G [tf, f/+i — 1] according to some 
pre-specified probability distribution which depends only on 
L[f] and feedback during that frame. 

Then, since L[f] is an i.i.d. process over frames, this 
algorithm is stationary. Furthermore, let {T*[f]}fLQ and 

be two processes corresponding to 

this randomized algorithm. It can be shown that these two 
processes are both i.i.d. over frames. By strong law of large 
numbers. 


lim 

F—loc F 


F-l 


Et*[/] 

/=o 


E[T*[f]] 


lim — 
F—ioc F 


F-ltf+T’U]-! 

E E 


f=0 t=tf 


= E 


tf+T'lf]-! 

E 

t=tf 


w.p.l, 

P) 

w.p.l. 
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Furthermore, it can be shown that the optimal solution to ([TJ- 
© can be achieved over this class of algorithms (see 0 for 
related results). That being said, there exists a randomized 
stationary algorithm such that 


E[T*[f]] = e*, 


E 




t — tf 


< 0 , 


(13) 

(14) 


where 9* is the optimal delay solving ©-©. As we shall see, 
this result plays an important role in our subsequent proof of 
optimality. Unfortunately, this algorithm is not implementable 
because the source has no knowledge on the distribution 
of packet length L[f]. Even if it has the knowledge, the 
computation complexity is very high because of the possibly 
large packet length set C. 


C. Key feature inequality 

We give a feature of our proposed online algorithm which 
is the key to the proof of optimality. Define T([/] as the system 
history up to frame /. Consider the algorithm that at the 
beginning of frame /, observes Q[f], L[f] and then chooses 
a sequence of PSD allocations P{t) G V so as to minimize 
(l4]i-(|5]l. Then, the policy must have achieved a smaller value 
on (IHi compared to that of best stationary policy from Section 
IIV-BI Mathematically, this can be expressed as the following 
inequality: 


E 


<E 


VT[f] + Q[f] E (PW-P) nflLif] 

t=tf 

VT*[f]+Q[f] E n[f],L[f] 

t=tf 


where the left-hand-side uses values T[f] and P{f) from the 
implemented policy, and the right-hand-side uses values T* [/] 
and P*{f) from the stationary policy. 

Taking expectation from both sides regarding L\f] gives 


E 

<E 


VT[f] + Q[f] E (Pit)-(3) 


n[f] 


t^+T*[/]-l 

VT*[f] + Q[f] E 

t=tf 


n[f] 


with probability 1, where Cq is a positive constant independent 
of V. 

Proof: First, it is obvious from the two lemmas in Section 
IIV-AI that the time average constraint is satisfied. The rest is 
devoted to proving the delay guarantee. 

Define the Lyapunov drift A[/] for the virtual queue Q[f] 
as follows 


A[/] = i(Q[/ + lF-Q[/n. 


Substitute the definition of Q[f 1] gives 

2 




tf+i-i 


t=tf 




E +Q[/] E 


min 

2 


Recall that the frame length T[f] is upper bounded by 

and P{t) < P|p|. Let Cq = ^ (-P|-p| +/?)) , it 

follows 


A[/]<Co+Q[/] E 

t — tf 

Now, we consider the following expected drift term A[/] plus 
penalty term VT[f]: 


E[A[f] + VT[f]\n[f]] 


<Co+E 

Qif] E iPit)-P) + VT[f] 

nn 


t—tf 


<Co + V6 

*, 



where the last inequality follows from (fTsT l. Thus, rearranging 
the terms gives 

E[A[f] + V{T[f]-e*)\n[f]]<Co. (16) 

Since Q[f] is deterministically bounded, A[/] is also deter¬ 
ministically bounded, thus, the following holds 


Using the fact that the stationary randomized policy is inde¬ 
pendent of history as well as (fOl l and (fT4li . 


E 


VT[f] + Q[f] E iPit)-P) 

t=tf 


nif] 


< ve* (15) 


D. Near optimal performance 

Theorem 1: The proposed algorithm satisfies the constraint 
and yields the time average delay satisfying 

limsup ^ E A 

F-^oc ^ ^ 


oo 


E 


E [A[/]^] 
F 


< (X. 


(17) 


Corollary 4.2 of IflTl states that if A[/] satisfies ( fTTl i. then, 
(fTbl l implies the following holds with probability 1: 


1 

limsup-^V(Tlf]-9*)<Co. 

Divide both sides by V and rearrange the terms gives the 
desired result. ■ 
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Appendix A 

Proof of optimality for (O 

Proof: First of all, for any < 0, since Rf{Pj) > 

0, Vj S {1, 2, • • • , \V\}, the best solution is not choosing any 
PSD allocation at all. This results in m[fc] = 0, Vfc < 0. Next, 
for A: > 0, we use induction to show m\k\ indeed satisfies the 
iteration (O. 

1) Base case fc = 1. Since K{ai,Pj) is a positive integer, 
(IT 2 I 1 gives m[l] = minj R{Pj). On the other hand, the 
best solution is to place one weight which has the least 
penalty R{Pj), which obviously implies m[l] satisfies 
the iteration (fT2l i. 

2) Suppose m[k] satisfies (fT2l i for all k G ,ko}, 

then, we want to prove m[fco + 1] also satisfies (fT2l i. We 
show that 

mlko + 11 < min Rf(Pj) 
iefi.-.l-PI} 

1^1 

-^'^(l){ai)m[ko-\-l - K{ai,Pj)] ( 18 ) 

and 

mlkn + 1] > min Rf(Pi) 

1-41 

+ '^(j){ai)m[ko-\-l-K{ai,Pj)] (19) 

i^l 

respectively. 

Proof of (fTSl l.- Consider the following specific allocation 
policy: Pick any Pj G V, and then adopt the policy 
which solves ([TOli- dTTI i for fe = feo + 1 — K{ai,Pj). By 
induction hypothesis, the resulting penalty of this policy 
is Rf{Pj) + 4){ai)m[kQ + 1 - K{ai,Pj)\. Since 
this is a specific policy, it follows, 

1-41 

m[fco + 1] < Rf{Pj) + X! 4>{ai)m[ko + 1 - Kiai, Pj)], 

i=l 


which implies (fTSl l. 

Proof of (fT^.- Suppose P(tf). - , P(f/+i — 

1) is a sequence of PSD allocations generated by the 
optimal policy minimizing (ITOli- dTTl i for k = fcp + 1. 
Given the event a{tf) and allocation P{tf) at time 
slot tf, we look at the remaining sequence of PSD 
allocations P{tf + 1), • • • , P(f/+i — 1). It can be 
regarded as a sequence generated by a specific policy 
under the constraint 

tj+i-i 

P, K{a{t), Pit)) >ko + l- K{aitf), P(f/)), 

t=tf+i 


which does not necessarily minimize when 

fc = fco + 1 — Kia{tf), P(f/)). Formally, this idea can 


be expressed as 


=E 


m[fco + 1] 

RfiPit)) : Y Kiait),Pit)) > fco + 1 

t=tf 


=E 


Py(P(f;))+E 


t=tf 

tf+l-1 


Y Mm)-- 

t=tf+i 


Y >ko + l-Kiaitf),Pitf)) 

t=tf+i 


Ti 


tf 


>E 


P/(P(f/))+ min 
■' P{t)GV 


E MPM-- 

t=tf+i 


*/+i-i 

E ^im^Pit)) > ko + l- K{aitf),Pitf)) 

t=tf+i 


Ti 


tf 


=E 


[PfiPitf)) + w[fco + 1 - K{aitf), P(f/))] 


where Ptf denotes all the system information up until 
time slot tf and the last equality follows from induction 
hypothesis. Finally, since P{tf ) also comes from a 
specific policy, it follows. 


m[fco + 1] > min PAPA 

+ E[m[fco + 1 - K{aitf),Pj)]] 

= min Pf (PA 

1-41 

+ E 4>iai)m[ko + 1 - Kiat, Pj)] 

i=l 

Overall, we proved the optimality. ■ 
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